Hi, I'm using the Apache XML-RPC code in a project and I had some
bad interoperability problems when using a non-Apache client.
Specifically, when I used the python client to send a binary of
about 700k, it gained about 10k in the decoding. This seems to be because
the decoder does not do anything to skip input characters that are not
valid input characters. In particular, section 6.8 of RFC 2045 states the
following:
The encoded output stream must be represented in lines of no more
than 76 characters each. All line breaks or other characters not
found in Table 1 must be ignored by decoding software. In base64
data, characters other than those in Table 1, line breaks, and other
white space probably indicate a transmission error, about which a
warning message or even a message rejection might be appropriate
under some circumstances.
The python encoder was indeed sending 76 character lines, which
seemed to be the problem.
Not wanting to spend a lot of time getting this thing going, I
replaced the base64 decoder with one that I had written and have done
interop testing with. The patch is included, and you guys are free to do
what you want with it.
Like the original decoder, it does not throw an exception on
invalid input, it instead tried to decode as best as it can. It may be
more appropriate here to throw an exception if the invalid character is
not a whitespace, i.e. change this:
// Skip over invalid characters.
if(x<0) {
invalid++;
continue;
}
to
// Skip over invalid characters.
if(x<0) {
if(!Character.isWhitespace(input.charAt(i))) {
throw new IOException("Invalid Base64 character found");
}
// Otherwise, mark it and continue
invalid++;
continue;
}
Anyway, you guys are free to do with this code as you please, but
I'm using it in the meantime to make my app work.
--
SPY My girlfriend asked me which one I like better.
pub 1024/3CAE01D5 1994/11/03 Dustin Sallings <[EMAIL PROTECTED]>
| Key fingerprint = 87 02 57 08 02 D0 DA D6 C8 0F 3E 65 51 98 D8 BE
L_______________________ I hope the answer won't upset her. ____________
Index: src/java/org/apache/xmlrpc/Base64.java
===================================================================
RCS file: /home/cvspublic/xml-rpc/src/java/org/apache/xmlrpc/Base64.java,v
retrieving revision 1.3
diff -r1.3 Base64.java
0a1,4
> // Copyright (c) 2001 Dustin Sallings <[EMAIL PROTECTED]>
> //
> // $Id: Base64.java,v 1.7 2001/04/03 07:59:28 dustin Exp $
>
3,64c7
< /*
< * $Header: /home/cvspublic/xml-rpc/src/java/org/apache/xmlrpc/Base64.java,v 1.3
2002/03/20 15:11:03 mpoeschl Exp $
< * $Revision: 1.3 $
< * $Date: 2002/03/20 15:11:03 $
< *
< * ====================================================================
< *
< * The Apache Software License, Version 1.1
< *
< * Copyright (c) 1999 The Apache Software Foundation. All rights
< * reserved.
< *
< * Redistribution and use in source and binary forms, with or without
< * modification, are permitted provided that the following conditions
< * are met:
< *
< * 1. Redistributions of source code must retain the above copyright
< * notice, this list of conditions and the following disclaimer.
< *
< * 2. Redistributions in binary form must reproduce the above copyright
< * notice, this list of conditions and the following disclaimer in
< * the documentation and/or other materials provided with the
< * distribution.
< *
< * 3. The end-user documentation included with the redistribution, if
< * any, must include the following acknowlegement:
< * "This product includes software developed by the
< * Apache Software Foundation (http://www.apache.org/)."
< * Alternately, this acknowlegement may appear in the software itself,
< * if and wherever such third-party acknowlegements normally appear.
< *
< * 4. The names "The Jakarta Project", "Tomcat", and "Apache Software
< * Foundation" must not be used to endorse or promote products derived
< * from this software without prior written permission. For written
< * permission, please contact [EMAIL PROTECTED]
< *
< * 5. Products derived from this software may not be called "Apache"
< * nor may "Apache" appear in their names without prior written
< * permission of the Apache Group.
< *
< * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR IMPLIED
< * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
< * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
< * DISCLAIMED. IN NO EVENT SHALL THE APACHE SOFTWARE FOUNDATION OR
< * ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
< * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
< * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
< * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
< * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
< * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
< * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
< * SUCH DAMAGE.
< * ====================================================================
< *
< * This software consists of voluntary contributions made by many
< * individuals on behalf of the Apache Software Foundation. For more
< * information on the Apache Software Foundation, please see
< * <http://www.apache.org/>.
< *
< * [Additional notices, if required by prior licensing conditions]
< *
< */
---
> import java.io.*;
67,74c10
< * This class provides encode/decode for RFC 2045 Base64 as defined by
< * RFC 2045, N. Freed and N. Borenstein. <a
< * href="http://www.ietf.org/rfc/rfc2045.txt";>RFC 2045</a>:
< * Multipurpose Internet Mail Extensions (MIME) Part One: Format of
< * Internet Message Bodies. Reference 1996
< *
< * @author Jeffrey Rodriguez
< * @version $Id: Base64.java,v 1.3 2002/03/20 15:11:03 mpoeschl Exp $
---
> * Base64 block encoder/decoder.
76,319c12,180
< public final class Base64
< {
< static private final int BASELENGTH = 255;
< static private final int LOOKUPLENGTH = 64;
< static private final int TWENTYFOURBITGROUP = 24;
< static private final int EIGHTBIT = 8;
< static private final int SIXTEENBIT = 16;
< static private final int SIXBIT = 6;
< static private final int FOURBYTE = 4;
< static private final int SIGN = -128;
< static private final byte PAD = (byte) '=';
< static private byte [] base64Alphabet = new byte[BASELENGTH];
< static private byte [] lookUpBase64Alphabet = new byte[LOOKUPLENGTH];
< //static private final Log log =
LogSource.getInstance("org.apache.commons.util.Base64");
<
< static
< {
< for (int i = 0; i < BASELENGTH; i++ )
< {
< base64Alphabet[i] = -1;
< }
< for (int i = 'Z'; i >= 'A'; i--)
< {
< base64Alphabet[i] = (byte) (i - 'A');
< }
< for (int i = 'z'; i>= 'a'; i--)
< {
< base64Alphabet[i] = (byte) (i - 'a' + 26);
< }
< for (int i = '9'; i >= '0'; i--)
< {
< base64Alphabet[i] = (byte) (i - '0' + 52);
< }
<
< base64Alphabet['+'] = 62;
< base64Alphabet['/'] = 63;
<
< for (int i = 0; i <= 25; i++ )
< lookUpBase64Alphabet[i] = (byte) ('A' + i);
<
< for (int i = 26, j = 0; i <= 51; i++, j++ )
< lookUpBase64Alphabet[i] = (byte) ('a'+ j);
<
< for (int i = 52, j = 0; i <= 61; i++, j++ )
< lookUpBase64Alphabet[i] = (byte) ('0' + j);
<
< lookUpBase64Alphabet[62] = (byte) '+';
< lookUpBase64Alphabet[63] = (byte) '/';
< }
<
< public static boolean isBase64( String isValidString )
< {
< return isArrayByteBase64(isValidString.getBytes());
< }
<
< public static boolean isBase64( byte octect )
< {
< //shall we ignore white space? JEFF??
< return (octect == PAD || base64Alphabet[octect] != -1);
< }
<
< public static boolean isArrayByteBase64( byte[] arrayOctect )
< {
< int length = arrayOctect.length;
< if (length == 0)
< {
< // shouldn't a 0 length array be valid base64 data?
< // return false;
< return true;
< }
< for (int i=0; i < length; i++)
< {
< if ( !Base64.isBase64(arrayOctect[i]) )
< return false;
< }
< return true;
< }
<
< /**
< * Encodes hex octects into Base64.
< *
< * @param binaryData Array containing binary data to encode.
< * @return Base64-encoded data.
< */
< public static byte[] encode( byte[] binaryData )
< {
< int lengthDataBits = binaryData.length*EIGHTBIT;
< int fewerThan24bits = lengthDataBits%TWENTYFOURBITGROUP;
< int numberTriplets = lengthDataBits/TWENTYFOURBITGROUP;
< byte encodedData[] = null;
<
<
< if (fewerThan24bits != 0)
< {
< //data not divisible by 24 bit
< encodedData = new byte[ (numberTriplets + 1 ) * 4 ];
< }
< else
< {
< // 16 or 8 bit
< encodedData = new byte[ numberTriplets * 4 ];
< }
<
< byte k = 0, l = 0, b1 = 0, b2 = 0, b3 = 0;
<
< int encodedIndex = 0;
< int dataIndex = 0;
< int i = 0;
< //log.debug("number of triplets = " + numberTriplets);
< for ( i = 0; i<numberTriplets; i++ )
< {
< dataIndex = i*3;
< b1 = binaryData[dataIndex];
< b2 = binaryData[dataIndex + 1];
< b3 = binaryData[dataIndex + 2];
<
< //log.debug("b1= " + b1 +", b2= " + b2 + ", b3= " + b3);
<
< l = (byte)(b2 & 0x0f);
< k = (byte)(b1 & 0x03);
<
< encodedIndex = i * 4;
< byte val1 = ((b1 & SIGN)==0)?(byte)(b1>>2):(byte)((b1)>>2^0xc0);
< byte val2 = ((b2 & SIGN)==0)?(byte)(b2>>4):(byte)((b2)>>4^0xf0);
< byte val3 = ((b3 & SIGN)==0)?(byte)(b3>>6):(byte)((b3)>>6^0xfc);
<
< encodedData[encodedIndex] = lookUpBase64Alphabet[ val1 ];
< //log.debug( "val2 = " + val2 );
< //log.debug( "k4 = " + (k<<4) );
< //log.debug( "vak = " + (val2 | (k<<4)) );
< encodedData[encodedIndex+1] =
< lookUpBase64Alphabet[ val2 | ( k<<4 )];
< encodedData[encodedIndex+2] =
< lookUpBase64Alphabet[ (l <<2 ) | val3 ];
< encodedData[encodedIndex+3] = lookUpBase64Alphabet[ b3 & 0x3f ];
< }
<
< // form integral number of 6-bit groups
< dataIndex = i*3;
< encodedIndex = i*4;
< if (fewerThan24bits == EIGHTBIT )
< {
< b1 = binaryData[dataIndex];
< k = (byte) ( b1 &0x03 );
< //log.debug("b1=" + b1);
< //log.debug("b1<<2 = " + (b1>>2) );
< byte val1 = ((b1 & SIGN)==0)?(byte)(b1>>2):(byte)((b1)>>2^0xc0);
< encodedData[encodedIndex] = lookUpBase64Alphabet[ val1 ];
< encodedData[encodedIndex + 1] = lookUpBase64Alphabet[ k<<4 ];
< encodedData[encodedIndex + 2] = PAD;
< encodedData[encodedIndex + 3] = PAD;
< }
< else if (fewerThan24bits == SIXTEENBIT)
< {
<
< b1 = binaryData[dataIndex];
< b2 = binaryData[dataIndex +1 ];
< l = (byte) (b2 & 0x0f);
< k = (byte) (b1 & 0x03);
<
< byte val1 = ((b1 & SIGN) == 0)?(byte)(b1>>2):(byte)((b1)>>2^0xc0);
< byte val2 = ((b2 & SIGN) == 0)?(byte)(b2>>4):(byte)((b2)>>4^0xf0);
<
< encodedData[encodedIndex] = lookUpBase64Alphabet[ val1 ];
< encodedData[encodedIndex + 1] =
< lookUpBase64Alphabet[ val2 | ( k<<4 )];
< encodedData[encodedIndex + 2] = lookUpBase64Alphabet[ l<<2 ];
< encodedData[encodedIndex + 3] = PAD;
< }
<
< return encodedData;
< }
<
< /**
< * Decodes Base64 data into octects
< *
< * @param binaryData Byte array containing Base64 data
< * @return Array containing decoded data.
< */
< public static byte[] decode( byte[] base64Data )
< {
< // handle the edge case, so we don't have to worry about it later
< if(base64Data.length == 0) { return new byte[0]; }
<
< int numberQuadruple = base64Data.length/FOURBYTE;
< byte decodedData[] = null;
< byte b1=0,b2=0,b3=0, b4=0, marker0=0, marker1=0;
<
< // Throw away anything not in base64Data
<
< int encodedIndex = 0;
< int dataIndex = 0;
< {
< // this sizes the output array properly - rlw
< int lastData = base64Data.length;
< // ignore the '=' padding
< while (base64Data[lastData-1] == PAD)
< {
< if (--lastData == 0)
< {
< return new byte[0];
< }
< }
< decodedData = new byte[ lastData - numberQuadruple ];
< }
<
< for (int i = 0; i < numberQuadruple; i++)
< {
< dataIndex = i * 4;
< marker0 = base64Data[dataIndex + 2];
< marker1 = base64Data[dataIndex + 3];
<
< b1 = base64Alphabet[base64Data[dataIndex]];
< b2 = base64Alphabet[base64Data[dataIndex +1]];
<
< if (marker0 != PAD && marker1 != PAD)
< {
< //No PAD e.g 3cQl
< b3 = base64Alphabet[ marker0 ];
< b4 = base64Alphabet[ marker1 ];
<
< decodedData[encodedIndex] = (byte)( b1 <<2 | b2>>4 ) ;
< decodedData[encodedIndex + 1] =
< (byte)(((b2 & 0xf)<<4 ) |( (b3>>2) & 0xf) );
< decodedData[encodedIndex + 2] = (byte)( b3<<6 | b4 );
< }
< else if (marker0 == PAD)
< {
< //Two PAD e.g. 3c[Pad][Pad]
< decodedData[encodedIndex] = (byte)( b1 <<2 | b2>>4 ) ;
< }
< else if (marker1 == PAD)
< {
< //One PAD e.g. 3cQ[Pad]
< b3 = base64Alphabet[ marker0 ];
<
< decodedData[encodedIndex] = (byte)( b1 <<2 | b2>>4 );
< decodedData[encodedIndex + 1] =
< (byte)(((b2 & 0xf)<<4 ) |( (b3>>2) & 0xf) );
< }
< encodedIndex += 3;
< }
< return decodedData;
< }
---
> public class Base64 extends Object {
>
> private static final char charmap[]={
> 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M',
> 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z',
> 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm',
> 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z',
> '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '+', '/'
> };
>
> /**
> * Get a base64 encode/decoder.
> */
> public Base64() {
> super();
> }
>
> /**
> * Encode some bytes to a base64 string.
> */
> public String encode(byte data[]) {
> // Go ahead and guess how big it needs to be.
> StringBuffer sb=new StringBuffer((data.length*4/3));
>
> int o=0;
> // Flip through the input and encode the shite.
> for(int i=0; i<data.length; i+=3) {
> int a, b, c, tmpa, tmpb;
>
> a=((int)data[i] & 0xff);
> sb.append(charmap[(a>>2)]);
> tmpa=((a&0x03)<<4);
>
> // If there's another byte, grab it and process it
> if(data.length > i+1) {
> b=((int)data[i+1] & 0xff);
> tmpb=(b>>4);
> sb.append(charmap[(tmpa|tmpb)]);
> tmpa=((b&0x0f)<<2);
> if(data.length>i+2) {
> c=((int)data[i+2] & 0xff);
> tmpb=((c&0xc0)>>6);
> sb.append(charmap[(tmpa|tmpb)]);
> sb.append(charmap[(c&0x3f)]);
> } else {
> sb.append(charmap[tmpa]);
> sb.append('=');
> }
> } else {
> // Only one byte in this block.
> sb.append(charmap[tmpa]);
> sb.append('=');
> sb.append('=');
> }
>
> o+=4;
> if( (o%76)==0) {
> sb.append("\r\n");
> }
> }
>
> return(sb.toString());
> }
>
> /**
> * Decode a string back into the bytes.
> */
> public byte[] decode(String input) {
> // Get enough room to store the output.
> int size=(input.length()*3/4);
> int insize=input.length();
> if(input.endsWith("=")) {
> size--;
> insize--;
> if(input.endsWith("==")) {
> size--;
> insize--;
> }
> }
> byte rv[]=new byte[size];
> int pos=0;
> int count=0;
> int packer=0;
> int invalid=0;
>
> for(int i=0; i<insize; i++) {
> int x=mapIndex(input.charAt(i));
>
> // Skip over invalid characters.
> if(x<0) {
> invalid++;
> continue;
> }
>
> // Count valid chars.
> count++;
>
> // Pack them.
> packer = packer << 6 | x;
>
> // Every four bytes, we've got three valid output bytes.
> if(count==4) {
> rv[pos++]=(byte)((packer >> 16)&0xFF);
> rv[pos++]=(byte)((packer >> 8)&0xFF);
> rv[pos++]=(byte)(packer&0xFF);
> count=0; packer=0;
> }
> }
>
> // Any remainders?
> if(count==2) {
> rv[pos++]=(byte)(( (packer << 12) >> 16)&0xFF);
> } else if(count==3) {
> rv[pos++]=(byte)(( (packer << 6) >> 16)&0xFF);
> rv[pos++]=(byte)(( (packer << 6) >> 8) & 0xFF);
> }
>
> // If there were any invalid characters, our size was wrong.
> if(invalid>0) {
> byte tmp[]=new byte[pos];
> System.arraycopy(rv, 0, tmp, 0, pos);
> rv=tmp;
> }
>
> return(rv);
> }
>
> /**
> * Is this character a valid Base64 character?
> *
> * @return true if this character is in our Base64 character map.
> */
> public boolean isValidBase64Char(char c) {
> return(mapIndex(c)>=0);
> }
>
> private int mapIndex(char c) {
> int rv=-1;
>
> for(int i=0; i<charmap.length && rv==-1; i++) {
> if(charmap[i]==c) {
> rv=i;
> }
> }
>
> return(rv);
> }
>
> public static void main(String args[]) throws Exception {
> File f=new File(args[0]);
> System.out.println("Working on " + f + " (" + f.length() + "
>bytes).");
> FileInputStream fis=new FileInputStream(f);
> byte data[]=new byte[(int)f.length()];
> int size=fis.read(data);
> fis.close();
> if(size!=f.length()) {
> throw new Exception("Didn't read all the data.");
> }
>
> Base64 b=new Base64();
> String tmp=b.encode(data);
>
> System.out.println(tmp);
>
> FileOutputStream fos=new FileOutputStream(args[1]);
> fos.write(b.decode(tmp));
> fos.close();
> }
>
Index: src/java/org/apache/xmlrpc/WebServer.java
===================================================================
RCS file: /home/cvspublic/xml-rpc/src/java/org/apache/xmlrpc/WebServer.java,v
retrieving revision 1.11
diff -r1.11 WebServer.java
684c684,685
< byte[] c = Base64.decode(line.substring(21).getBytes());
---
> Base64 base64=new Base64();
> byte[] c = base64.decode(line.substring(21));
Index: src/java/org/apache/xmlrpc/XmlRpc.java
===================================================================
RCS file: /home/cvspublic/xml-rpc/src/java/org/apache/xmlrpc/XmlRpc.java,v
retrieving revision 1.22
diff -r1.22 XmlRpc.java
642c642,643
< value = Base64.decode(cdata.getBytes());
---
> Base64 base64=new Base64();
> value = base64.decode(cdata);
769c770,771
< this.write(Base64.encode((byte[]) obj));
---
> Base64 base64=new Base64();
> this.write(base64.encode((byte[]) obj));
Index: src/java/org/apache/xmlrpc/XmlRpcClient.java
===================================================================
RCS file: /home/cvspublic/xml-rpc/src/java/org/apache/xmlrpc/XmlRpcClient.java,v
retrieving revision 1.9
diff -r1.9 XmlRpcClient.java
139c139,140
< auth = new String(Base64.encode((user + ':' + password)
---
> Base64 base64=new Base64();
> auth = new String(base64.encode((user + ':' + password)
Index: src/java/org/apache/xmlrpc/applet/SimpleXmlRpcClient.java
===================================================================
RCS file:
/home/cvspublic/xml-rpc/src/java/org/apache/xmlrpc/applet/SimpleXmlRpcClient.java,v
retrieving revision 1.4
diff -r1.4 SimpleXmlRpcClient.java
247c247,248
< writer.write(Base64.encode((byte[]) what));
---
> Base64 base64=new Base64();
> writer.write(base64.encode((byte[]) what));
642c643,644
< value = Base64.decode(cdata.getBytes());
---
> Base64 base64=new Base64();
> value = base64.decode(cdata);
